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We introduce a growing network model in which a new node attaches to a randomly-selected node, 
as well as to all ancestors of the target node. This mechanism produces a sparse, ultra-small network 
where the average node degree grows logarithmically with network size while the network diameter 
equals 2. We determine basic geometrical network properties, such as the size dependence of the 
number of links and the in- and out-degree distributions. We also compare our predictions with 
real networks where the node degree also grows slowly with time — the Internet and the citation 
network of all Physical Review papers. 
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I. INTRODUCTION 

Many networks in nature and technology are sparse, 
i.e., the average node degree is much smaller than the 
total number of nodes N 0, Q • Widely studied classes 
of networks, such as regular grids, random graphs, and 
scale-free networks, are maximally sparse, as the aver- 
age node degree remains finite as TV — > oo. However, in 
other examples of real sparse networks, such as the In- 
ternet, the average node degree grows, albeit very slowly, 
with system size. Motivated by this observation, we in- 
troduce a simple network growth mechanism of copying 
that naturally generates sparse networks in which the av- 
erage node degree diverges logarithmically with system 
size. We dub these log-networks. Related results appear 
in previous investigations of related models 0, 3 , while 
models where a slowly increasing ratio of links to nodes 
is imposed externally have also been considered 0, |(| . 

To motivate the copying mechanism for log-networks 
let us recall the growing network with redirection (GNR) 
■ The GNR is built by adding nodes according to the 
following simple rule. Each new node initially selects an 
earlier "target" node at random. With a specified prob- 
ability a link from the new node to the target node is 
created; with a complementary probability, the link is 
re-directed to the ancestor node of the target. Although 
the target node is chosen randomly, the redirection mech- 
anism generates an effective preferential attachment be- 
cause a high-degree node is more likely to be the ancestor 
of a randomly-selected node. By this feature, redirection 
leads to a power-law degree distribution for the network. 
The GNR thus provides an appealingly simple mecha- 
nism for preferential attachment, as well as an extremely 
efficient way to simulate large scale- free networks 

The growing network with redirection is a simplifica- 
tion of a previous model 9] which was proposed to mimic 
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the copying of links in the world-wide web. In this web 
model, a new node links to a randomly-chosen target 
node and also to its ancestor nodes (subject to a con- 
straint on the maximum number of links created) . In the 
context of citations, copying is (regrettably) even more 
natural, as it is easier merely to copy the references of a 
cited paper, rather than to look at the original references 
|lfj| . As the literature grows, the copying mechanism will 
necessarily lead to later publications having more refer- 
ences than earlier publications. 

In the following sections, we analyze a growing network 
model with copying (GNC). We consider a model with 
no global bound on the number of links emanating from 
a new node. We shall see that this simple copying mech- 
anism generates log-networks. We will use the master 
equation approach to derive basic geometric properties 
of the network. We then compare our prediction about 
the logarithmic growth of the average degree with data 
from Physical Review citations. 



II. GNC MODEL 

We now define the GNC model precisely. The network 
grows by adding nodes one at a time. A newly-introduced 
node randomly selects a target node and links to it, as 
well as to all ancestor nodes of the target node (Fig-QJ. 



new node 



node 



FIG. 1: Illustration of the growing network with copying 
(GNC). The time order of the nodes is indicated. Initial links 
are solid and links to ancestor nodes are dashed. Later links 
partially obscure earlier links. The new node initially attaches 
to random target node 4, as well as to its ancestors, 1 and 3. 
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If the target node is the initial root node, no addi- 
tional links are generated by the copying mechanism. If 
the newly-introduced node were to always choose the root 
node as the target, a star graph would be generated. On 
the other hand, if the target node is always the most re- 
cent one in the network, all previous nodes are ancestors 
of the target and the copying mechanism would give a 
complete graph. Correspondingly, the total number of 
links L jv in a network of N nodes can range from N — 1 
(star graph) to N(N— 1)/2 (complete graph). Notice also 
that the number of outgoing links from each new node 
(the out-degree) can range between 1 and the current 
number of nodes. 



III. NETWORK STRUCTURE 

We now study geometric properties of the GNC model 
by the master equation approach. We determine how 
the total number of links L grows with N, as well as the 
in-degree, out-degree, and the joint in/out-degree distri- 
butions. 



A. Total Number of Links 

Let L(N) be the average value of the total number of 
links in a network of N nodes. If a newly-introduced node 
selects a target node with j ancestors, then the number 
of links added to the network will be l+j. Therefore the 
average total number of links satisfies 

L(N+l) = L(7V) + 1(^(1 + jQ )) 

a 

, ^ L(N) 
= L(N) + 1 + ^. (1) 

The factor iV _1 in the first line assures that a target 
node a is selected uniformly from among all N nodes, 
and we obtain the second line by employing the sum rule 

Dividing Eq. by N + 1 gives 

L(N+1) L(N) _ 1 
N + 1 N~ ~ N+l' 

and then summing both sides from 1 to N — 1 gives the 
solution 

L(N) = N (Hn — 1) (2) 

Here Hn = ^ 1 n^ 1 is the harmonic number. (For 
concreteness, we assume that the network starts with a 
single node, so that L(l) = 0.) Using the asymptotics 
of the harmonic numbers \H\ we find 

L(N)=N]nN-N(l--y) + ±- 1 ± N + ..., (3) 



where 7 = 0.57721566 ... is the Euler constant. The lead- 
ing asymptotic behavior of NhiN can also be obtained 
more easily from Eq. by taking the continuum approx- 
imation and solving the resulting differential equation. 




FIG. 2: Distribution of the number of links Pjv(L) for 10 5 
realizations of a GNC of N = 1000 sites. Shown are both the 
raw data and the result of averaging the data over a 1% range. 
For this value of N, the mean number of links is 6485.56, while 
Eq. gives 6485.47. . . . 

Thus we conclude that the average degree of the net- 
work grows logarithmically with the system size; that is, 
the copying mechanism generates a log-network. This 
simple phenomenon is one of our major results. 

We now briefly discuss the probability distribution of 
the total number of links Pn(L) for a network of N nodes. 
Simulations show that the distribution is asymmetric and 
quite broad (Fig. To understand the origin of the 
asymmetry, notice that both the extreme cases of the 
star graph [L = N — 1) and the complete graph (L = 
N(N — l)/2) each occur with probability 

because each new node must select one specific target 
node. Therefore the distribution of the total number of 
links vanishes much more sharply near the lower cutoff. 

Near the peak however, the distribution Pjv(L) is sym- 
metric about the average. More precisely, when the de- 
viation from the average L(N) = ^2 L LPm{L) is of the 
order of S(iV) = = L ( N )} 2 p n(L), the distribu- 

tion approaches a symmetric Gaussian shape. The value 
of the standard deviation as N — > 00 is 

Z{N)^CN, C = y/2 - 7r 2 /6 = 0.595874 . . . , (4) 

as derived in Appendix [X] The relative width of the 
distribution is measured by the standard deviation S(iV) 
divided by the average L(N); this ratio approaches zero 
as (lniV) -1 when N — ► 00, so that fluctuations die out 
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slowly. This slow decay of fluctuations explains why the 
distribution (Fig. |3J) remains wide for large N and why 
it looks asymmetric near the peak. 

The GNC model can be extended to allow for wider 
copying variability. For example, instead of linking to 
one initial random target node, we can link to m ran- 
dom initial targets. Further, we can link to each target 
node with probability p and to each of the corresponding 
ancestors with probability q. For this general (m,p,q) 
model, the analog of Eq. is 0] 

L(N + 1) = L(N) + ^(J2(P + 93a)) (5) 

which reduces to 4h = mp + mqj^ in the continuum 
approximation. The asymptotic growth of the average 
total number of links crucially depends on the parameter 
mq: 

! mP N for mq < 1 ; 
1 — mq 
mpNlnN for mq = 1; ( 6 ) 
oc N mq for mq>l. 

Thus incomplete copying leads to an average node degree 
that is independent of N when mq < 1, while marginal 
logarithmic dependence is recovered when mq = 1. There 
is also a pathology for mq > 2, as the number of links 
in the network would exceed that of a complete graph 
with the same number of nodes. In this case, it is not 
possible to accommodate all the links specified by the 
copying rule without having a multigraph, i.e., allowing 
for more than one link between a given pair of nodes. 

B. Comparison with Empirical Data 

We now present empirical data from the citation net- 
work of Physical Review to test whether the average de- 
gree of these networks grows with time, and if so, whether 
the growth is consistent with log-networks. Data from 
all issues of Physical Review journals is available, en- 
compassing a time span of 110 years ^3|. From this 
data, we have the following evidence that citations may 
be described as a log-network. Specifically, the average 
number of references in the reference list of each Physical 
Review paper grows systematically with time and is con- 
sistent with a linear increase (Fig. . Additionally, the 
number of Physical Review papers published in a given 
year roughly grows exponentially with time |l3j|. Thus 
the cumulative number of Physical Review papers up to 
a given year also grows exponentially. As a result, the 
number of references should grow logarithmically with 
the total number of available papers. This behavior is 
reasonably consistent with the data of Fig. [3J 

In a related vein, the average number of coauthors per 
paper has grown slowly with time, due in part, to the 
growing trend for collaborative research and the contin- 
uing ease of long-distance scientific interaction. While 



o 




1890 1910 1930 1950 1970 1990 2010 

year 

FIG. 3: Average number of references in the reference list 
of Physical Review papers published in each year (o). Also 
shown as a smooth curve is the logarithm of the cumulative 
number of Physical Review papers that were published up to 
each year. The value 5 is subtracted from the latter data to 
make the two datasets lie in the same range. 

co-authorship and other collaboration networks have re- 
cently been investigated (see, e.g., 0,0,0), the anal- 
ysis has primarily been on network properties at a fixed 
time. There is, however, one study of the number of 
mathematics papers with 1, 2, and more authors since 
1940 0| . This data shows that the fraction of singly- 
authored papers is decreasing systematically, while the 
number of multiple-authored papers is steadily growing. 
Thus it should be interesting to track the time depen- 
dence of the number of co-authors in scientific publica- 
tions from the current studies of collaboration networks. 

Interestingly, the Internet and the world wide web ex- 
hibit certain similarities with log- networks. For example, 
the total number of links exceeds the total number of 
nodes in the world wide web by about an order of mag- 
nitude 0,E1- Similarly for the Internet, specifically for 
the Autonomous Systems (AS) graph, the average num- 
ber of links per node is also growing slowly but system- 
atically with time [2(j ■ Qualitatively these behaviors are 
consistent with our expectations from log- networks. It 
is not still possible to reach definitive conclusions about 
the precise growth rate on N since the available data for 
the AS graph 0] covers a time period when the total 
number of ASes has increased only by a factor of 4 (from 
N = 3060 in 1997 to N = 12155 in 2001). 

C. In-degree distribution 

By its very construction, the links of the GNC network 
are directed and thus there is an in-degree i and an out- 
degree j for each node (Fig. |3}, and thus two distinct 
corresponding degree distributions. In this subsection, 
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we study the in-degree distribution. 




j=5 



FIG. 4: A node with in-degree (number of incoming links) 
i = 4, out-degree (number of outgoing links) j — 5, and total 
degree k — 9. 

Let Pi{N) be the average number of nodes with in- 
degree i in a network consisting of N total nodes. This 
distribution satisfies 

Pi(N+l) = PiW-lllpiW + jjPi-iW+Sw (7) 

The loss term accounts for the following two processes: 
(a) either a node of in-degree i, or (b) any of its i daughter 
nodes was chosen as the target. Either of these processes 
leads to the loss of a node with in-degree i. The total 
loss rate of Pi(N) is thus (i + 1)/N. The gain term is 
explained similarly, and the last term on the right-hand 
side of Eq. Q describes the effect of the introduction 
of a new node with no incoming links. Finally, notice 
that Eqs. Q hold for i < N. When i — N, there is no 
longer a loss term and the master equation reduces to 
P N (N + 1) = P N -i(N) = 1. This accounts for the fact 
that the root node is necessarily linked to all other nodes 
and therefore there is one node with degree N — 1 in a 
network of TV nodes. 

We compute the in-degree distribution by induction. 
Solving for the first few Pi(N) for small i directly we 
find a simple form for the general case that we then check 
solves the master equation {7J. We thus find 

N 

Pi W = , xn , x ,v for i < N - 1, (8) 
(t + l)(i + 2) 

while Pi(N) = for i > N. 

The asymptotic i~ 2 decay agrees with the logarithmic 
divergence for the average node degree from the previ- 
ous section, and perhaps explains the proliferation of 
exponent values close to 2 for the in-degree distribu- 
tion that are observed in empirical studies of collabo- 
ration networ ks IT3 . Il5l ITr3 | and in the world wide web 
[Til \l$L l2lL I22I l23j L In particular, a comprehensive study 
by by Broder et al. 18] reports an exponent value 2.09, 
while a recent work by Donato et al. jn| (relying on the 
WebBase project at Stanford |24|) quotes an exponent 
value 2.1. 



D. Out-degree distribution 

To determine the out-degree distribution, it is helpful 
to think of the network as a genealogical tree, as illus- 
trated in Fig. (see also Q for this construction). Ini- 
tially the network consists of one root node. Subsequent 




FIG. 5: Genealogical tree representation of the network of 
Fig. with nodes arranged in layers left to right according 
to their out-degree. The initial layer contains only the root 
node. The number of nodes in subsequent layers increases as 
the network grows. The initial links are shown as solid arrows 
and the copied links as dashed arrows. 



nodes that attach to the root node will have out-degree 
1 and lie in the first layer. Similarly, a new node that 
attaches to a node with out-degree 1 lies in the second 
layer. By the copying mechanism, nodes in this second 
layer also link to the root and therefore have out-degree 
2. Nodes in the n th layer directly attach to a node in the 
(n — l) st layer and, by virtue of copying, also attach to 
one node in every previous layer. Thus n th layer nodes 
have out-degree n. We now use this genealogical tree 
picture to determine the out-degree distribution of the 
network. 

Let Qj (N) be the average number of nodes with out- 
degree j in a network consisting of N nodes. By definition 
Qo(N) = 1. On the other hand, the number of nodes 
with out-degree j > 1 grows each time a node with out- 
degree j — 1 is selected as the target node. The out-degree 
distribution thus satisfies the master equation 



Q j (N + l)=Q j (N) + ± 



tS-i 



(AO- 



(9) 



This equation applies even for j = if we set Q_i(iV) = 0. 
Using the recursive nature of these equations, we first 
solve for Qi(N), then Q2{N), etc, and ultimately the 
out-degree distribution for all j. This procedure gives 

Q 1 (N+1)= ~ l — ( 10 ) 



l<mi<...<Tn,-<iV 



mi X 



X TO, 



Equivalently, we can recast the j-fold sums into simple 
sums, although the results look less neat. For example, 



Q 2 (N 



(H N f i4 2) 



where H 



(2) 

JV 



E 



N 



The asymptotic behaviors of 



iijv , 1 and other generalized harmonic numbers are 



r(2) 

known |ll| . and the resulting asymptotics of the out- 
degree distribution are 



h(N+l) = H N = ln7V + 7 + 
h(N+l) = i(lnA0 2 + 7 lniV + l 



1 



1 



2N UN 2 



1 



7 ~ T 
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and analogous results hold for Qj(N) for larger j. 

If we merely want to establish the leading asymptotic 
behavior, we can replace the summation in (|10fl by inte- 
gration. This then leads to the simple result 



(In NY 



(11) 



Alternatively, we can derive this result within a contin- 
uum approach by replacing finite differences by deriva- 
tives in the large- N limit of Eq. ©. The procedure re- 
casts the discrete master equations into the differential 
equations 



dQj 
~dN 



1 

n' 



whose solution is indeed l|llfl . 

The Poisson form of the out-degree distribution con- 
tradicts the commonly presumed power-law form. There 
is previous literature by Broder et al. that suggested that 
the out-degree distribution has a power-law tail, with ex- 
ponent close to 2.7 0- However, this work also noted 
that a power-law is not a good fit to the data and that 
the out-degree distribution may possibly follow a Pois- 
son distribution. In fact, the analysis of Ref. 19] that is 
based on more recent data on the structure of the web 
[24| convincingly shows that a power law does not fit the 
out-degree distribution. 



E. Joint degree distribution 

We define the joint degree distribution Nij(N) as the 
average number of nodes with in-degree i and out-degree 
j in a network of N nodes. The in- and out-degree distri- 
butions can then be distilled from the joint distribution 



viaP,(iV) 



E, N itj (N) and Q 3 (N) = N itj (N). Fur- 



thermore, the average number of nodes with total degree 
k is simply given by N k (N) = J2i+j= 
The joint degree distribution satisfies 



N id (N+l) = N hj (N) + ± N^ ld (N) - -j^- Nij(N) 



(12) 



which is an obvious generalization of the governing equa- 
tions Q and © for the separate in- and out-degree dis- 
tributions. Because of the presence of the last out-degree 
term on the right-hand side of equation (|12fl . the scaling 
of the joint degree distribution with system size does not 
hold - Nij(N) 7^ Nriij. Therefore we cannot reduce 
(|12fl to an TV-independent recursion. 

Nevertheless, Eqs. (|12f) still have the important sim- 
plifying feature of being recursive and thus soluble in an 
inductive fashion. Thus, for example, for i = we have 

NN 0ij (N + 1) = (N- 1) N 0>j (N) + Qj-i(N) 
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FIG. 6: The in- (o), out- (A), and total (v) degree distri- 
butions for 10 4 realizations of a network of N = 10 4 nodes. 
Notice that there is always one node with total degree equal 
to 1 and one node with in- and total degree equal to TV. The 
smooth curve that follows in in-degree data is the asymptotic 
prediction JSJ, while the curve that follows the out-degree 
data is the asymptotic prediction 1111 . 



from which 



N, 



1 N 

o,i(^ + l) = ^E Qo-iiM) 



(13) 



M=l 



For i > 1, we rewrite Eqs. (11211 in the form 



NN id (N + l) = (N-i-l)Nij(N)+iNi- ld (N)- (14) 



Now the substitution 

reduces (|14H to the constant-coefficient recursion 

A hJ (N + 1) = A itj (N) + At-uiN) 
that allows us to express Aij via Ai—ij: 

N 

A h3 (N + l)= £ Ai-ij{M). 

M=l 



(15) 



(16) 



From (J]])) we find A ,j(N + 1) = N- 1 N 0d (N + 1) 
which, in conjunction with (|13f) . gives 



N 



A 0J (N + 1)= Qj-i( M ) 



(17) 



M=l 



Therefore starting with Aqj from Eq. (|17fl . we find all 
the A iy j via Eq. (|lu[l . The final result is 

A,j(N)= Yl Qi-i(m) (18) 

l<m <...<m i <A r 
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Equations (fTOft . l(T5|l. (fTSj) give the full solution for the 
joint degree distribution. 

While the complete solution is cumbersome, it can be 
simplified as N — > oo. In this limit, we can first replace 
the factor ^(n)^ by N^ 1 in Eq. JT^. Additionally, 
we can replace the summation in Eq. I|18[l by integration. 
These two replacements are justified when i <C y/~N. Fi- 
nally, using (|f f \ and after some algebra, we find the lead- 
ing behavior 



and more generally 



(In AT) 



(i-i)! 



(In AT) 



(7-1)! 



i+l 



(19) 



Because the Poisson form the out-degree distribution 
holds only when j <C In N, the generalized Poisson form 
(|f f I) for the joint degree distribution is also valid only for 
j < In TV. 

Finally, although the total degree distribution Nk(N) 
does not satisfy a closed equation, we can obtain this 
distribution indirectly. When k is of the order of In N or 
smaller, we can use (|19H to find Nk(N). The situation 
in the range k 3> IniV is even simpler: In this region, 
the total degree distribution essentially coincides with 
the in-degree distribution and therefore Nk(N) — > N k~ 2 
(Pig- B ■ 



IV. SUMMARY 

We introduced a growing network model that is based 
on node addition plus a simple copying mechanism - 
the GNC — that leads to an average node degree grow- 
ing logarithmically with the total number of nodes N. 
This feature may account for the intriguing phenomenon 
observed in many real networks that the number of links 
increases slightly faster than the number of nodes. Copy- 
ing arises naturally in the context of citations; a not un- 
typical scenario is that an author will be familiar with a 
few primary references, but may simply copy secondary 
references from primary ones. 

We solved the underlying master equations for the 
GNC model and showed that the in-degree distribution 
is a power-law over its entire range, while the out-degree 
distribution is asymptotically Poissonian. The total de- 
gree distribution is consequently a hybrid of the power- 
law and Poisson forms. There is, on average, one node 
with total degree equal to 1, and there is always one node 
— the root — that has in-degree equal to N — 1. Thus 
the node degree ranges from one to N — 1. Since the 
distribution of L has a width that scales linearly with N 
while L(N) grows as N\nN, fluctuations in node degree 
are appreciable even for very large networks. Finally, 
each node is connected to the root, so that the network 
diameter equals 2, independent of N. 



From long-term Physical Review publication data, the 
average number of references per paper (the out degree) 
grows slowly with the total literature size, consistent with 
the logarithmic growth predicted by the GNC model. 
However, this growth in the GNC model is not robust 
when parameters that quantify the extent of copying are 
varied. The apparent logarithmic growth for the average 
number of references per paper in the Physical Review 
is thus a bit surprising and it will be worthwhile to test 
whether logarithmic growth arises in a wider range of 
empirical networks. 
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APPENDIX A: FLUCTUATIONS 

In this appendix, we find the variance in the distribu- 
tion of the number of links Pn(L). We start by comput- 
ing the first two moments of the out-degree distribution, 
u N = (j) = T,jjQj(N)/N and v N = (j 2 ). We then use 
these results to derive the variance of Pjv(i). 

To determine ujv and vn, we can in principle use 
Eq. (|I0I) . However, a direct approach is more useful. 
Starting with Nun = (%2j), wc find that adding a new 
node leads to the recursion relation 



(N + l)u N+1 = (l+ja+^j 

= 1 + u N + Nu N , 



(Al) 



which is nothing but Eq. Q. In a similar manner, we 
derive a recursion relation for Nvn = (^2j 2 ) 

(N + l)v N+1 = ((I+j Q ) 2 +]Tj 2 

= I + 2u N + v N + Nv N , 

which reduces to 



VN+l — VN + 



1 



N+l N + l 

The variance <J 2 (N) = vn — {un) 2 therefore satisfies 

I I 



(A2) 



a\N + l) = a\N) + 



N + l (N - 
From this simple recursion we get 



1 



a 2 (N) = H N -H 



(2) 
N ■ 



(A3) 



(A4) 



The relative magnitude of fluctuations die out slowly, as 
the standard deviation, a(N) ~ VlniV divided by the 
average it at ~ InN, approaches zero as (In N)^ 1 / 2 . 
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Consider now the average number of links in the net- 
work L(N) — j) — Nujy and the corresponding sec- 
ond moment L2(N) = (L 2 N ). After the addition of a new 
node, the second moment changes according to 



L 2 (N+l) = ((l + j a +J2j 



= l + 2u N + v N + 2Nu N +\ [ l + — j L 2 (N) 

Now we use L(N) = Q3i) — Nun to write the square 
of Eq. (|A1|) in the form 



The homogeneous part of (|A5|) suggests seeking a solu- 
tion of the form E 2 (7V) = N(N+1)S N . This substitution 
recasts JUJ into S N+1 -S N = [(N +1)(N + 2)]~ 1 a 2 (N). 
This is an exact discrete first derivative for Sn. Hence 
S N equals £f -1 [(M + 1 ){ M + 2)}- 1 a 2 (M). Thus the 



variance is 



JV-l 



S 2 (iV) = N(N + 1) 



M= 



a 2 (M) 



' t (M + l)(M + 2) 



(A6) 



L(N+lf = M +-J L(^) 2 + ( WA ,) 2 + 2(A^+1) MW + 1, 

and then subtracting this from the previous equation, the 
variance £ 2 (iV) = L 2 (N) — L(N) 2 satisfies 



E 2 (A^ + 1) = ( 1 + ^ ) Z 2 (N) + a 2 (N) (A5) 



Finally, by substituting a 2 (M) = X^xmO 1 — j 2 ) from 
(|A4(I into (|A6|I . and changing the order of the two sums, 
we find that £ 2 (A0 (2 - ± n 2 )N(N + 1) as iV — > oo. 
This leads to the asymptotic expression for the standard 
deviation given in Eq. QJ. 
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